Multimodal training of maintenance and assembly skills based on augmented reality

The training of technicians in the acquisition of new maintenance and assembly tasks is an important factor in industry. As the complexity of these tasks can be enormous, the training of technicians to acquire the necessary skills to perform them efficiently is a challenging point. However, traditional training programs are usually highly theoretical and it is difficult for the trainees to transfer the acquired theoretical knowledge about the task to the real task conditions, or rather, to the physical performance of the task. In addition, traditional training programs are often expensive in terms of effort and cost. Previous research has shown that Augmented Reality is a powerful technology to support training in the particular context of industrial service procedures, since instructions on how to perform the service tasks can be directly linked to the machine parts to be processed. Various approaches exist, in which the trainee is guided step-by-step through the maintenance task, but these systems act more as guiding systems than as training systems and focus only on the trainees’ sensorimotor capabilities. Due to the increasing complexity of maintenance tasks, it is not sufficient to train the technicians’ execution of these tasks, but rather to train the underlying skills—sensorimotor and cognitive—that are necessary for an efficient acquisition and performance of new maintenance operations. All these facts lead to the need for efficient training systems for the training of maintenance and assembly skills which accelerate the technicians’ learning and acquisition of new maintenance procedures. Furthermore, these systems should improve the adjustment of the training process to new training scenarios and enable the reuse of existing training material that has proven its worth. In this thesis a novel concept and platform for multimodal Augmented Reality-based training of maintenance and assembly skills is presented. This concept includes the identification of necessary sub-skills, the training of the involved skills, and the design of a training program for the training of maintenance and assembly skills. Since procedural skills are considered as the most important skills for maintenance and assembly operations, they are discussed in detail, as well as appropriate methods for improving them. We further show that the application of Augmented Reality technologies and the provision of multimodal feedback—and vibrotactile feedback in particular—have a great potential to enhance skill training in general. As a result, training strategies and specific accelerators for the training of maintenance and assembly skills in general and procedural skills in particular are elaborated. Here, accelerators are concrete methods used to implement the pursued training strategies. Furthermore, a novel concept for displaying location-dependent information in Augmented Reality environments is introduced, which can compensate tracking imprecisions. In this concept, the pointercontent metaphor of annotating documents is transferred to Augmented Reality environments. As a result, Adaptive Visual Aids are defined which consist of a tracking-dependent pointer object and a tracking-independent content object, both providing an adaptable level and type of information. Thus,